Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation

Neural dialogue models suffer from low-quality responses when interacted in practice, demonstrating difficulty generalization beyond training data. Recently, knowledge distillation has been used to successfully regularize the student by transferring teacher. However, teacher and are trained on same dataset tend learn similar feature representations, whereas most general should be found through differences. The finding of is further hindered unidirectional distillation, as obey may discard some that truly but refuted To this end, we propose a novel framework, where learning more line with idea reaching consensus, i.e., common beneficial different yet all datasets diversified partners. Concretely, task divided into group subtasks number students. Each assigned one subtask not only optimized allocated also imitates multi-view representation aggregated other students (i.e., peers), which induces capture among alleviates over-fitting subtasks. enhance generalization, extend bidirectional encourages its peers co-evolve exchanging complementary each other. Empirical results analysis demonstrate our framework effectively improves model without sacrificing efficiency.